33 research outputs found
Neuro-Visualizer: An Auto-encoder-based Loss Landscape Visualization Method
In recent years, there has been a growing interest in visualizing the loss
landscape of neural networks. Linear landscape visualization methods, such as
principal component analysis, have become widely used as they intuitively help
researchers study neural networks and their training process. However, these
linear methods suffer from limitations and drawbacks due to their lack of
flexibility and low fidelity at representing the high dimensional landscape. In
this paper, we present a novel auto-encoder-based non-linear landscape
visualization method called Neuro-Visualizer that addresses these shortcoming
and provides useful insights about neural network loss landscapes. To
demonstrate its potential, we run experiments on a variety of problems in two
separate applications of knowledge-guided machine learning (KGML). Our findings
show that Neuro-Visualizer outperforms other linear and non-linear baselines
and helps corroborate, and sometime challenge, claims proposed by machine
learning community. All code and data used in the experiments of this paper are
available at an anonymous link
https://anonymous.4open.science/r/NeuroVisualizer-FDD
Predictive Learning with Heterogeneity in Populations
University of Minnesota Ph.D. dissertation. October 2017. Major: Computer Science. Advisor: Vipin Kumar. 1 computer file (PDF); x, 119 pages.Predictive learning forms the backbone of several data-driven systems powering scientific as well as commercial applications, e.g., filtering spam messages, detecting faces in images, forecasting health risks, and mapping ecological resources. However, one of the major challenges in applying standard predictive learning methods in real-world applications is the heterogeneity in populations of data instances, i.e., different groups (or populations) of data instances show different nature of predictive relationships. For example, different populations of human subjects may show different risks for a disease even if they have similar diagnosis reports, depending on their ethnic profiles, medical history, and lifestyle choices. In the presence of population heterogeneity, a central challenge is that the training data comprises of instances belonging from multiple populations, and the instances in the test set may be from a different population than that of the training instances. This limits the effectiveness of standard predictive learning frameworks that are based on the assumption that the instances are independent and identically distributed (i.i.d), which are ideally true only in simplistic settings. This thesis introduces several ways of learning predictive models with heterogeneity in populations, by incorporating information about the context of every data instance, which is available in varying types and formats in different application settings. It introduces a novel multi-task learning framework for problems where we have access to some ancillary variables that can be grouped to produce homogeneous partitions of data instances, thus addressing the heterogeneity in populations. This thesis also introduces a novel strategy for constructing mode-specific ensembles in binary classification settings, where each class shows multi-modal distribution due to the heterogeneity in their populations. When the context of data instances is implicitly defined such that the test data is known to comprise of contextually similar groups, this thesis presents a novel framework for adapting classification decisions using the group-level properties of test instances. This thesis also builds the foundations of a novel paradigm of scientific discovery, termed as theory-guided data science, that seeks to explore the full potential of data science methods but without ignoring the treasure of knowledge contained in scientific theories and principles
A Data-Driven Approach to Full-Field Damage and Failure Pattern Prediction in Microstructure-Dependent Composites using Deep Learning
An image-based deep learning framework is developed in this paper to predict
damage and failure in microstructure-dependent composite materials. The work is
motivated by the complexity and computational cost of high-fidelity simulations
of such materials. The proposed deep learning framework predicts the
post-failure full-field stress distribution and crack pattern in
two-dimensional representations of the composites based on the geometry of
microstructures. The material of interest is selected to be a high-performance
unidirectional carbon fiber-reinforced polymer composite. The deep learning
framework contains two stacked fully-convolutional networks, namely, Generator
1 and Generator 2, trained sequentially. First, Generator 1 learns to translate
the microstructural geometry to the full-field post-failure stress
distribution. Then, Generator 2 learns to translate the output of Generator 1
to the failure pattern. A physics-informed loss function is also designed and
incorporated to further improve the performance of the proposed framework and
facilitate the validation process. In order to provide a sufficiently large
data set for training and validating the deep learning framework, 4500
microstructural representations are synthetically generated and simulated in an
efficient finite element framework. It is shown that the proposed deep learning
approach can effectively predict the composites' post-failure full-field stress
distribution and failure pattern, two of the most complex phenomena to simulate
in computational solid mechanics